Apparatus and method for upconverting content data

ABSTRACT

A signal processing apparatus includes an upconversion processing unit upconverting content data stored in a data storage unit, and an upconversion control unit determining an upconversion-target component and the upconversion processing order of components based on component attribute information such as a component type or a tag value of each component of the content data such as elementary streams.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to apparatus and methods for upconvertingcontent data including plural components.

2. Description of the Related Art

Hitherto, various techniques have been researched regarding upconversionprocessing for realizing more realistic and powerful images and sounds,such as a technique for enhancing video image quality and sound quality.

Along with an increase in display screen size and resolution, a demandfor products equipped with the image quality enhancing technique wouldgrow in the future. For example, a technique for converting astandard-definition (SD) video image to a high-definition (HD) videoimage (resolution conversion) through classification/adaptive processinghas been under study. Further, various techniques have been proposed forenhancing the image quality, such as a technique for generating ahigh-quality video image through γ correction (gamma correction) orcontour correction.

Japanese Patent Application Laid-Open No. 07-321662 discusses atechnique that combines time-part processing and frequency-partprocessing to increase processing accuracy. Japanese Patent ApplicationLaid-Open No. 2004-246526 discusses a technique for achieving ahigh-accuracy processing result which more precisely corresponds to realevents, in consideration of the real events where data is obtained.

Further, a demand for products equipped with the sound quality enhancingtechnique has been increased. Japanese Patent Application Laid-Open No.09-261071 discusses a technique for converting a format of multi-bitaudio data having a sampling frequency fS and extracted from a compactdisc (CD) into 1-bit audio data having a sampling frequency m*fS (m isan integer). This technique enables reproduction of even a subtle soundor nuance that could not be realized by the CD.

There are two methods for upconversion processing, that is, a method forprocessing content data concurrently with reproduction of the data and amethod of processing content data over time, not concurrently with thereproduction.

For example, in the case of raising video image quality through dynamicgamma correction, content data can be processed almost concurrently withreproduction. On the other hand, in the case of raising video imagequality through resolution conversion or raising sound quality throughformat conversion, in some cases, content data could not be processedconcurrently with reproduction because of its high processing load.

For example, in the case of converting a format of multi-bit audio dataextracted from a CD into 1-bit audio data as described above, if asystem throughput is not so high, processing takes several times as longas the reproduction time of music. In this case, content data may betemporarily stored in a recording medium and then gradually upconverted.In the case of gradually upconverting data in this way, if a processingtime is long, the following problem occurs. That is, when a user desiresto reproduce content data being processed, the content data cannot bereproduced.

According to the moving pictures experts group 2 (MPEG-2) standard, astandard adopted for digital broadcasting standards and a package mediasuch as digital versatile disc (DVD), plural elementary streams (ESs)are multiplexed. To be specific, plural components, for example, videoimages, sounds, and additional information are multiplexed.

According to this standard, plural video components are multiplexed toenable a user to view multi-angle video images (video images taken withplural cameras at different angles) at the same time or by freelyswitching the angles. Further, it is possible to support variouslanguages by multiplexing plural audio components.

At the time of upconverting content data obtained by multiplexing pluralcomponents (elementary streams), if all components are upconverted inparallel, a high processing load is imposed on the system. However, itis supposedly unnecessary to upconvert all components at the same timein many cases.

For example, if a user wants to view only a main video image ofmulti-angle video images, it is preferred to enhance a quality of themain video image ahead of (prior to) sub video images. If a user wantsto view a music program with a high sound quality, it is preferred toenhance a sound quality ahead of (prior to) an image quality.

SUMMARY OF THE INVENTION

Embodiments of the present invention can provide a signal processingapparatus and a signal processing method which can efficiently upconvertcontent data composed of plural components.

According to an aspect of the present invention, a method ofupconverting content data including two or more components capable ofreproduction by reproduction apparatus. The method includes generatingupconversion control information for said content data, and upconvertingone or more components of the content data in accordance with thegenerated upconversion control information.

According to exemplary embodiments of the present invention, at the timeof upconverting content data including plural components, the pluralcomponents can be upconverted in efficient order. This enhancesconvenience for users.

Further features and aspects of the present invention will becomeapparent from the following detailed description of exemplaryembodiments with reference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate exemplary embodiments, features,and aspects of the invention and, together with the description, serveto explain the principles of the invention.

FIG. 1 is a block diagram illustrating a configuration of a signalprocessing apparatus according to a first exemplary embodiment of thepresent invention.

FIG. 2 is a block diagram illustrating an internal configuration of anupconversion processing unit of FIG. 1.

FIG. 3 illustrates an example of content information stored in a memoryof FIG. 1.

FIG. 4 is a flowchart illustrating an upconversion processing procedureof the first exemplary embodiment.

FIG. 5 is a flowchart illustrating a procedure for determining anupconversion target component and processing order.

FIG. 6 illustrates an example screen for checking the progress ofupconversion processing.

FIG. 7 illustrates a time period necessary for upconversion processing.

FIG. 8 illustrates an example of content information according to amodified example of the first exemplary embodiment.

FIG. 9 is a flowchart illustrating a procedure for determining theupconversion processing order of components by a user.

FIG. 10 illustrates an example of a graphical user interface (GUI)screen for setting the upconversion processing order.

FIG. 11 illustrates an example of updated content information.

FIG. 12 illustrates an example of an automatically displayed guidescreen according to a second exemplary embodiment of the presentinvention.

FIG. 13 illustrates an example of content information according to athird exemplary embodiment of the present invention.

FIG. 14 illustrates an example of genre list information of the thirdexemplary embodiment.

FIG. 15 is a block diagram illustrating a configuration of a signalprocessing apparatus according to a fourth exemplary embodiment of thepresent invention.

FIG. 16 illustrates an example of viewing history information collectedby a viewing history collecting unit of FIG. 15.

FIG. 17 is a flowchart illustrating a processing procedure of the fourthexemplary embodiment.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Various exemplary embodiments, features, and aspects of the inventionwill be described in detail below with reference to the drawings.

First Exemplary Embodiment

FIG. 1 is a block diagram illustrating a configuration of a signalprocessing apparatus 100 according to a first exemplary embodiment ofthe present invention. In FIG. 1, a broadcasting signal received by anantenna (or a cable) is input to a tuner unit 101. The tuner unit 101performs demodulation, error correction, or other processing on thebroadcasting signal received from the antenna to generate and outputdata (TS data) in a transport stream (TS) format.

A separating unit 102 extracts data of a channel selected from TS datacorresponding to plural channels output from the tuner unit 101. Then,the separating unit 102 outputs video data and audio data to a decoder103, and outputs electronic program guide (EPG) data and broadcast datato a stream data processing unit 116.

The decoder 103 decodes the audio data from the separating unit 102 andoutputs the decoded data to an audio output unit 105. The audio outputunit 105 performs amplification and digital-analog conversion of theaudio data which is decoded by the decoder 103, and then outputs theresultant data to a speaker 107. In addition, the decoder 103 decodesthe video data from the separating unit 102 and outputs the data to ascreen synthesizing unit 104.

The stream data processing unit 116 stores EPG data and broadcast dataoutput from the separating unit 102 in a memory 117 via a bus. Thescreen synthesizing unit 104 reads the EPG data or broadcast data storedin the memory 117 in response to a user's instruction input from aremote controller 119 to compose an EPG screen or a data broadcastingscreen. Then, the screen synthesizing unit 104 combines the data withvideo data from the decoder 103 and outputs the resultant data to avideo output unit 106. The video output unit 106 displays a screencomposed by the screen synthesizing unit 104 on a display unit 108.

A storage/reproduction control unit 109 stores TS data from the tunerunit 101 in a data storage unit 110 or reads TS data from the datastorage unit 110 to reproduce the data in response to a user'sinstruction input from the remote controller 119. Further, thestorage/reproduction control unit 109 reproduces TS data stored in thedata storage unit 110 in the upconversion processing. Thestorage/reproduction control unit 109 outputs the TS data read from thedata storage unit 110 to the separating unit 102 in the reproduction.

An upconversion processing unit 113 (processing means) performsupconversion processing on the video data and audio data output from thedecoder 103 and outputs the resultant data to an encoding/multiplexingunit 111.

FIG. 2 is a block diagram illustrating an internal configuration of theupconversion processing unit 113 of FIG. 1. The data output from thedecoder 103 is input to the upconversion processing unit 113 via aselector 301. To be specific, video components (video elementarystreams) output from the decoder 103 are input to a high-quality imagegeneration unit 302 to enhance the image quality, and audio components(audio elementary streams) output from the decoder 103 are input to ahigh-quality sound generation unit 303 to enhance the sound quality.

If the data output from the decoder 103 includes components other thanupconversion-target components, the data is input to theencoding/multiplexing unit 111 without being upconverted. Anupconversion control unit 112 (upconversion control means) controlsoperations of the selector 301 according to component attributeinformation (“component type” or “tag value”).

The quality enhanced video data from the high-quality image generationunit 302 and the quality enhanced audio data from the high-quality soundgeneration unit 303 are input to the encoding/multiplexing unit 111.

In FIG. 1, the encoding/multiplexing unit 111 encodes the upconvertedvideo data and audio data, and multiplexes the resultant data togetherwith broadcast data output from the stream data processing unit 116. TheTS data multiplexed by the encoding/multiplexing unit 111 is stored inthe data storage unit 110 through the storage/reproduction control unit109.

The upconversion control unit 112 controls an upconverting operation ofthe upconversion processing unit 113. To be specific, the upconversioncontrol unit 112 stores content information regarding content data (TSdata) stored in the data storage unit 110 in the memory 117, and then,performs registration, update, and deletion of the content informationon a content basis.

The content information is registered when the storage/reproductioncontrol unit 109 stores the contents in the data storage unit 110 orprograms the data storage unit 110 to store the contents. The contentinformation can be stored in the data storage unit 110 together with thecontent data. The control operation of the upconversion control unit 112is described in detail below.

Here, the upconversion processing in the present invention refers toquality enhancement processing (enhancing image quality and soundquality) that would be practically difficult to execute in real time,that is, the quality enhancement processing that requires time longerthan an actual time length of content. For example, the upconversionprocessing assumes the quality enhancement processing that imposes ahigh processing load such as processing for reproducing details withreference to plural video frames and converting video data into ahigh-definition video image.

In such cases, a complicated algorithm is used, so that it is difficultto perform upconversion processing in real time. Also, in the case ofexecuting complicated sound-quality enhancement processing such asreproducing sounds faithfully to the original sounds and reproducingrealistic sounds, it is difficult to perform the upconversion processingin real time.

To that end, in the exemplary embodiments of the present invention,content data (TS data) stored in the data storage unit 110 is graduallyupconverted. That is, the content data (TS data) read from the datastorage unit 110 by the storage/reproduction control unit 109 issupplied to the upconversion processing unit 113 via the separating unit102 and the decoder 103 and then upconverted.

A user interface (UI) control unit 114 acquires various types of GUIdata (image data, text data, etc.) from the memory 117 according to auser's instruction input from the remote controller 119 or a systemoperation status. Then, the UI control unit 114 controls the screensynthesizing unit 104 to compose various GUI screens.

A main control unit 115 executes control over blocks in the signalprocessing apparatus 100. A light receiving unit 118 receives remotecontrol signals input from the remote controller 119 and transfers thesignals to each control unit.

FIG. 3 illustrates an example of content information stored in thememory 117 of FIG. 1. In FIG. 3, a content information table includesprogram identifying information, component attribute information,information about the upconversion processing order, and informationwhether upconversion processing is completed.

The program identifying information is used to identify content data(program) stored in the data storage unit 110. This information iscomposed of “service_id”, “event_id”, “start_time” and “event_name”included in EPG data. The “service_id” is an identification number foridentifying a broadcasting station (or channel number). The “event_id”is an identification number for identifying a program. The “start_time”represents starting date and time of a target program, and the“event_name” represents a program name.

A “tag value” is described in a program map table (PMT) and assigned toeach component of the content data. The Association of Radio Industriesand Businesses (ARIB) standard STD-B10 specifies the “tag value” forcomponent as “component_tag”. For example, the ARIB STD-B10 assigns a“tag value” in the range from 0*00 to 0*0F to video components, a “tagvalue” in the range from 0*10 to 0*2F to audio components, and a “tagvalue” in the range from 0*30 to 0*7F to other components.

The “component type” is described in an event information table (EIT)and represents attribute of each component of the content data (mainvideo data, sub video data 1, sub video data 2, main audio data, subaudio data 1, sub audio data 2, etc.). The “component type” is definedby a descriptor of “text_char” in the EIT. Further, “Gr.ID” is definedin a multiview TV service that allows a user to view plural multi-anglevideo images at the same time or to switchably view the images. The“Gr.ID” is defined by “component_group_id” of a component groupdescriptor in the EIT.

The upconversion control unit 112 determines the upconversion processingorder according to the component attribute information (“component type”or “tag value”) of components in the content data. For example, theupconversion processing order of components of the program A isdetermined as: “main video data (default)”, “main audio data (default)”,“sub video data 1”, and “sub audio data 1”.

In this case, the processing order is determined so that video dataprecedes audio data and a main component precedes any sub component.Further, the “sub video data 2” and “sub audio data 2” are not subjectedto upconversion processing because of low priorities, and theirprocessing order is not determined. However, the above processing orderis an example and the present invention is not limited thereto. Aprocedure for determining the upconversion processing order is describedbelow.

The information in the upconverted/not upconverted column is a flagrepresenting whether upconversion processing for each component iscompleted. If the upconversion processing is completed, “0*01” is set.On the other hand, if the upconversion processing has not beencompleted, “0*00” is set. In the present exemplary embodiment,upconversion processing is intended for video data and audio data onlyand the other data (EPG data or broadcast data) is not subjected toupconversion processing, so that “0*FF” is set.

FIG. 4 is a flowchart illustrating the upconversion processing procedureof the first exemplary embodiment. In step S401, the upconversioncontrol unit 112 starts control for the upconversion processing when asystem operation availability falls below a predetermined thresholdvalue. In the present exemplary embodiment, if an operation load of theresources (a central processing unit (CPU)) is low, the content datastored in the data storage unit 110 will be upconverted. However, theupconversion control unit 112 can start control for the upconversionprocessing in response to a user's instruction to start the upconversionprocessing.

In step S402, the upconversion control unit 112 reads contentinformation of the content data as an upconversion target from thememory 117, and determines the upconversion-target component and theupconversion processing order. The operation in step S402 is describedin detail later with reference to FIG. 5.

Next, in step S403, the storage/reproduction control unit 109 reproducesthe content data stored in the data storage unit 110. In step S404, theseparating unit 102 separates the reproduced content data into the videodata, the audio data, and the other data. In step S405, the decoder 103decodes the video data and audio data output from the separating unit102.

Next, in step S406, the upconversion processing unit 113 upconverts theupconversion-target components of the video data and audio data outputfrom the decoder 103.

In step S407, the encoding/multiplexing unit 111 encodes each componentoutput from the upconversion processing unit 113. Next, in step S408,the encoding/multiplexing unit 111 multiplexes the encoded componenttogether with components from the stream data processing unit 116 (EPGdata or broadcasting data etc.) to generate TS data.

In step S409, the storage/reproduction control unit 109 stores themultiplexed content data (TS data) in the data storage unit 110. In stepS410, the upconversion control unit 112 updates the content informationin the memory 117 so that “upconverted” is set for upconvertedcomponents. In this way, the above-mentioned upconversion processing isexecuted on all content data until the whole content data is“upconverted”. Then, a series of processes is completed.

FIG. 5 is a detailed flowchart illustrating the processing fordetermining an upconversion-target component and the upconversionprocessing order in step S402 of FIG. 4. In step S501, the upconversioncontrol unit 112 reads content information stored in the memory 117 andacquires “start_time” of each content.

Then, in step S502, the upconversion control unit 112 specifies contentdata having the earliest “start_time” as upconversion-target contentdata. However, if the specified content data has been alreadyupconverted or includes no upconversion-target component, the contentdata having the next earliest “start_time” is specified as anupconversion target.

Next, in step S503, the upconversion control unit 112 acquires an EIT inthe content information of the specified content data and analyzes thecomponent attribute information (“component type” or “tag value”) ofeach component in the content data. Then, in step S504, the upconversioncontrol unit 112 determines the upconversion-target component and theprocessing order for each component based on the component attributeinformation.

In the present exemplary embodiment, as illustrated in FIG. 3, theupconversion processing order is set to the order of “main video data”,“main audio data”, “sub video data 1”, and “sub audio data 1” (in orderof increasing “tag value”). As for content data compliant with amultiview TV service, the processing order is set to the order ofincreasing a value of “Gr.ID” and the order of increasing a componenttag value in the same “Gr.ID”.

Here, “main video data”, “main audio data”, “subvideo data 1”, and “subaudio data 1” are set as an upconversion target, and “sub video data 2”,“sub audio data 2”, and “other data” are not set as an upconversiontarget, but the present invention is not limited thereto. For example,all video components and audio components can be set as an upconversiontarget, or only components corresponding to “main video data” and “mainaudio data” can be set as an upconversion target.

Next, in step S505, the upconversion control unit 112 updates thecomponent information to register the determined processing order.

FIG. 6 illustrates an example screen for checking the progress of theupconversion processing. If a user issues an instruction to display ascreen for checking the progress of the upconversion processing with theremote controller 119, a screen is composed by the UI control unit 114and the screen synthesizing unit 104 as illustrated in FIG. 6.

The remote controller 119 can include a dedicated button for “displayupconversion level”, or “display upconversion level” may be added tomenu items. In FIG. 6, the upconversion level indicates the progress ofthe upconversion processing on content data (program).

In the present exemplary embodiment, the following four states can bedisplayed. That is, if all upconversion-target components are alreadyupconverted, “⊚: recommended!” is displayed. If only componentscorresponding to “main video data” and “main audio data” areupconverted, “∘: best” is displayed. Further, if only components of“main video data” are upconverted, “Δ: almost completed” is displayed.If the upconversion processing is not yet executed, “×: unprocessed” isdisplayed.

A user can select and reproduce desired content data on the screen. Inthe case of reproducing and viewing a program ranked as “∘: best”, thecontrol unit can execute the upconversion processing on targetcomponents for which upconversion is not yet complete when the user isviewing the program. In this case, it is conceivable that upconversionprocessing is completed on all components and the upconversion level ischanged to “⊚: recommended!” while the user is viewing the program.

In the present exemplary embodiment, the content data (program) aresequentially upconverted one by one, but plural programs can beupconverted at a time. If plural programs are upconverted in parallel,the plural programs can be all ranked as “∘: best” or “Δ: almostcompleted”.

FIG. 7 illustrates a time period necessary for the upconversionprocessing. Referring to FIG. 7, a case of starting upconversionprocessing for four components, “main video data”, “main audio data”,“sub video data 1”, and “sub audio data 1” from time t0 is described.

As illustrated in FIG. 3, according to the first exemplary embodiment,the components are upconverted in the order of “main video data”, “mainaudio data”, “sub video data 1”, and “sub audio data 1”. In this case,during a period from time t0 to time t1, the upconversion level is “×:unprocessed”. At time t1, the upconversion processing for “main videodata” is completed, and the upconversion level is shifted to “Δ: almostcompleted”.

Then, at time t2, the upconversion processing for “main audio data” iscompleted, and the upconversion level is shifted to “∘: best”. At timet4, the upconversion processing for “sub video data 1” is completed, andat time t5, the upconversion processing for “sub audio data 1” iscompleted. Then, the upconversion level is shifted to “⊚: recommended!”.

In contrast, according to a comparative example 1 which is a firstvariant in accordance with the invention, the components are upconvertedin the order of “main video data”, “sub video data 1”, “main audiodata”, and “sub audio data 1”. At time t1, “main video data” iscompleted, and the upconversion level is shifted to “Δ: almostcompleted”. After that, at time t3, the upconversion processing for “subvideo data 1” is completed, but the upconversion level is not changed.

At time t4, the upconversion processing for “main audio data” iscompleted, and the upconversion level is shifted to “∘: best”. Then, attime t5, the upconversion processing for “sub audio data 1” iscompleted, and the upconversion level is shifted to “⊚: recommended!”.

Further, according to a comparative example 2 which is a second variantin accordance with the present invention, the components, “main videodata”, “sub video data 1”, “main audio data”, and “sub audio data 1” areupconverted in parallel. In this case, during a period from time t0 totime t5, the upconversion level is kept at “×: unprocessed”. Then, attime t5, the upconversion processing for all of “main video data”, “subvideo data 1”, “main audio data”, and “sub audio data 1” is completed,and the upconversion level is shifted to “⊚: recommended!”.

These examples are described on the assumption that the ending times arenot different between the case of upconverting the four components oneby one and the case of upconverting the four components in parallel, butthe ending time may vary in some degree.

As described above, the first exemplary embodiment of the presentinvention is the quickest for the upconversion level to reach “∘: best”(at time t2) compared to the comparative examples 1 and 2. If theupconversion of “main video data” and “main audio data” is completed, auser can view contents in sets with “main video data” and “main audiodata” although not enjoying the multiview TV service.

Most of users might generally view contents in sets with “main videodata” and “main audio data”, so it is effective to determine theupconversion processing order of components as in the first exemplaryembodiment of the present invention. In this way, preferentiallyupconverting the components having high probability of being viewed by auser allows the user to enjoy viewing contents upconverted at theearlier time.

In the present exemplary embodiment, the upconversion processing isintended for video data and audio data only, but the present inventionis not limited thereto. For example, components corresponding to databroadcasting may be subjected to the upconversion processing.

In the present exemplary embodiment, upconverted components are encodedand multiplexed again and then, converted into a TS data format andstored, but the data format is not limited thereto. For example,upconverted components can be stored in a reproducible form withoutbeing encoded nor multiplexed. In this case, separation processing anddecoding processing can be omitted in reproducing the data.

Further, instead of upconverting all of stored content data, the contentdata more suitable to user's preferences can be preferentiallyupconverted based on user's preference information.

Modified Example of First Exemplary Embodiment

In the first exemplary embodiment, the upconversion control unit 112determines the upconversion processing order so as to sequentiallyupconvert components of content data one by one, based on componentattribute information (“component type” or “tag value”) of eachcomponent. According to a modified example of the first exemplaryembodiment, a video component and an audio component are processed inparallel.

FIG. 8 illustrates an example of content information according to themodified example of the first exemplary embodiment. In FIG. 8, in theupconversion processing order of components of the program A, “mainvideo data (default)” and “main audio data (default)” are first , and“sub video data 1” and “sub audio data 1” second, for example.

Further, “sub video data 2” and “sub audio data 2” are upconvertedthirdly. In this case, the processing order is determined in the orderof increasing “tag value” and the main data precedes the sub data. Iflow-priority “sub video data 2” and “sub audio data 2” are not subjectedto the upconversion processing, the processing order of “sub video data2” and “sub audio data 2” is not determined.

In the modified example of the first exemplary embodiment, the selector301 of FIG. 2 inputs a video component and an audio component outputfrom the decoder 103 to the high-quality image generation unit 302 andthe high-quality sound generation unit 303 in parallel, respectively.Then, the image quality enhancement processing in the high-quality imagegeneration unit 302 and the sound quality enhancement processing in thehigh-quality sound generation unit 303 are executed in parallel.

Second Exemplary Embodiment

According to a second exemplary embodiment of the present invention, auser can determine a processing order of components ofupconversion-target content data (program). The following description isfocused on different points from the first exemplary embodiment.Components similar to those in the first exemplary embodiment aredenoted by the same reference numerals and thus detail description isnot repeated.

FIG. 9 is a flowchart illustrating how a user sets an upconversionprocessing order of components. Referring to FIG. 8 and FIG. 1 used inthe first exemplary embodiment, a procedure for setting the upconversionprocessing order in making a recording reservation of contents or inchanging reserved contents is described next. Here, the main controlunit 115 issues all instructions for executing a series of processes toeach processing unit.

In step S801, the UI control unit 114 displays a recordingreservation/changing screen on the display unit 108 in response to auser's operation from the remote controller 119.

In step S802, the main control unit 115 determines whether a userrequests to set the processing order of upconversion-target componentsin content data (program) that is reserved to record. If the user doesnot enter a request (NO in step S802), the processing advances to stepS804.

In step S804, the main control unit 115 makes a reservation forrecording or changes settings to update content information of thecontent data (program) that is reserved to record. If the user newlyreserves a program to record, the main control unit 115 newly registersthe content information of the target content data (program).

On the other hand, if the user enters a request to set the processingorder of upconversion-target components (YES in step S802), theprocessing advances to step S803. In step S803, the UI control unit 114displays a GUI screen for setting the upconversion processing order (seeFIG. 10). When the user sets the processing order of upconversion-targetcomponents on the displayed GUI screen, the processing advances to stepS804. In step S804, the main control unit 115 updates the contentinformation.

FIG. 10 illustrates an example of the GUI screen for setting theupconversion processing order. As apparent from FIG. 10, pluralupconversion-target components (“main video data”, “main audio data”,“sub video data 1”, and “sub audio data 1”) are displayed in defaultorder from top. The default processing order is determined beforehand ina manner similar to the first exemplary embodiment, for example.

The user can change the processing order of components by operating acursor control key of the remote controller 119 to move a focus totarget components and pressing the enter key. Further, the user canupconvert components other than the upconversion-target components (“subvideo data 2”, “sub audio data 2”, etc.) by operating the remotecontroller 119. Components that are beyond a display capacity of the GUIscreen can be displayed along with movement of the focus.

FIG. 11 illustrates an example of the content information updated instep S804 of FIG. 9. As understood from FIG. 11, a “user setting” columnindicating whether a user sets the upconversion order is set. If theuser changes the processing order of upconversion-target components onthe GUI screen, “0*01” is displayed in the “user setting” column;otherwise, “0*00” is displayed in the “user setting” column.

In FIG. 11, the processing order of “main video data” and “main audiodata” is interchanged in “program A” and setting of the processing orderof components is changed, so that “0*01” is displayed in the “usersetting” column of “program A”. Further, since the processing order ofcomponents of “program B” is not changed, “0*00” is displayed in the“user setting” column of “program B”.

FIG. 9 illustrates an example that the GUI screen for setting theupconversion processing order is displayed if a user enters a request toset the processing order of components. However, a guide screen can beautomatically displayed based on predetermined conditions. For example,as described in the first exemplary embodiment, if the upconversionprocessing is automatically started, the guide screen for setting theupconversion processing order is automatically displayed at thebeginning of the upconversion processing.

FIG. 12 illustrates an example guide screen which is automaticallydisplayed in the second exemplary embodiment. In FIG. 12, a GUI messageindicating that the upconversion processing order can be set isdisplayed. The guide screen automatically disappears after the elapse ofa predetermined period. If a user selects “YES” on the guide screen byoperating the remote controller 119, the UI control unit 114 displaysthe GUI screen of FIG. 10.

If the processing order of components of upconversion-target contentdata (program) is already changed by a user (“0*00” is set in the “usersetting” column), the guide screen can be set not to be displayed. Inthis case, only when the processing order of components is not yetchanged by the user (“0*01” is set in the “user setting” column), theguide screen is displayed. Alternatively, the guide screen can bedisplayed only if upconversion-target content data (program) is not aprogram reserved to record by the user but automatically recordedaccording to preference information.

As described above, according to the second exemplary embodiment, a usercan set the processing order of components of upconversion-targetcontent data (program). As a result, desired components can beupconverted earlier, and the apparatus can perform flexible upconversionprocessing according to user's requests.

In the second exemplary embodiment, components are sequentiallyupconverted one by one, but as described in the modified example of thefirst exemplary embodiment, the video component and the audio componentcan be processed in parallel.

Third Exemplary Embodiment

A third exemplary embodiment of the present invention differs from thefirst exemplary embodiment in a way of determining the processing orderof components of upconversion-target content data (program). Thefollowing description is focused on different points from the firstexemplary embodiment. Components similar to those in the first exemplaryembodiment are denoted by the same reference numerals and thus detaildescription is not repeated.

In the third exemplary embodiment, genre information of content data isused in executing the processing in step S504 of FIG. 5 (determinationof an upconversion-target component and the upconversion processingorder) described in the first exemplary embodiment. The configuration ofa signal processing apparatus according to the third exemplaryembodiment is identical to the first exemplary embodiment illustrated inFIG. 1 and thus is not described in detail.

However, content information stored in the memory 117 includesinformation about a genre of each content data (program). The memory 117also stores genre list information referenced in determining theupconversion-target component and the upconversion processing order.

FIG. 13 illustrates an example of content information according to thethird exemplary embodiment. As understood from FIG. 13, a “genre” columnindicating genre information of each content data (program) is set. Thegenre of “program C” is set to “0*0601”, and the genre of “program D” isset to “0*0000”. In this example, the genre information is described ina “content_nibble_level1” descriptor and a “content_nibble_level2”descriptor of the EIT.

FIG. 14 illustrates an example of genre list information according tothe third exemplary embodiment. The memory 117 stores in advance a tableof genre list information which prescribes the processing order ofupconversion-target components in association with each genre asillustrated in FIG. 14. For example, the processing order of componentscorresponding to the genre “cinema (foreign film)” is set to the orderof “main video data”, “main audio data (JP: Japanese)”, “sub video data1”, “sub video data 2”, and “sub audio data 1 (ENG: English)”, and theprocessing order of the sixth and subsequent components is determinedaccording to “tag value”.

The upconversion control unit 112 refers to content information storedin the memory 117 to obtain genre information of the upconversion-targetcontent data. Subsequently, the upconversion control unit 112 reads thegenre list information of FIG. 14 and refers to the upconversionprocessing order corresponding to the genre information of theupconversion-target content data. Then, the upconversion control unit112 sets the processing order of components and registers the order inthe content information illustrated in FIG. 13.

For example, as illustrated in FIG. 13, genre “0*0601” of “program C”corresponds to “cinema (foreign film)”. As illustrated in FIG. 14, theprocessing order of components specified to the genre “cinema (foreignfilm)” is set to the order of “main video data”, “main audio data (JP:Japanese)”, “sub video data 1”, “sub video data 2”, and “sub audio data1 (ENG: English)”. Therefore, as illustrated in FIG. 13, the processingorder of components of “program C” is set to the order of “main videodata”, “main audio data (JP: Japanese)”, “sub video data 1”, “sub videodata 2”, and “sub audio data 1 (ENG: English)”.

As illustrated in FIG. 14, the processing order of sixth and subsequentcomponents is determined according to “tag value”. Since “tag value:0*12” of “sub audio data 2” of “program C” is larger than “tag value:0*11” of “sub audio data 1” at the fifth position in the processingorder, “sub audio data 2” is set to the sixth position in the processingorder.

Further, genre “0*0000” of “program D” corresponds to “news”. Asillustrated in FIG. 14, the processing order of components specified tothe genre “news” is set to the order of “main video data”, “main audiodata”, and “sub audio data”. Hence, as illustrated in FIG. 13, theprocessing order of components of “program D” is set to the order of“main video data”, “main audio data”, and “sub audio data (1)”. In thiscase, the other components (for example, “sub video data 1”, “sub videodata 2” and “sub audio data 2”) are not subjected to the upconversionprocessing.

As described above, according to the third exemplary embodiment, anupconversion-target component and the upconversion processing order aredetermined based on a genre of content data, so that the upconversionprocessing is controlled so as to be more suitable for features ofcontents.

Therefore, in the case of a music program that might put emphasis on asound quality rather than an image quality, for example, the soundquality of components of “main audio data” are preferentially enhancedand a user can listen to high-quality audio content earlier. In the caseof a news program that might put emphasis on an image quality ratherthan a sound quality, the image quality of components of “main videodata” are preferentially enhanced and a user can view high-quality videocontent earlier.

In the third exemplary embodiment, components are sequentiallyupconverted one by one, but as described in the modified example of thefirst exemplary embodiment, the video component and the audio componentcan be processed in parallel.

Fourth Exemplary Embodiment

Similar to the third exemplary embodiment, a fourth exemplary embodimentof the present invention differs from the first exemplary embodiment ina way of determining the processing order of components ofupconversion-target content data (program). The following description isfocused on different points from the first exemplary embodiment.Components similar to those in the first exemplary embodiment aredenoted by the same reference numerals and thus detail description isnot repeated.

In the fourth exemplary embodiment, viewing history information of auser (past reproduction behavior) is used in executing the processing instep S504 of FIG. 5 (determination of an upconversion-target componentand the upconversion processing order) described in the first exemplaryembodiment.

FIG. 15 is a block diagram illustrating a configuration of a signalprocessing apparatus 100 according to the fourth exemplary embodiment ofthe present invention, in contrast to FIG. 1. A difference between FIG.1 and FIG. 15 is that the signal processing apparatus 100 of the fourthexemplary embodiment includes a viewing history collecting unit 120.

The viewing history collecting unit 120 collects viewing historyinformation about a program (content) selected by a user operating aremote controller 119, and stores (memorizes) the collected informationin a memory 117. The viewing history collecting unit 120 also collectsinformation about a reproduction condition of a program selected by areservation, for example, as well as programs actually selected andviewed by a user.

FIG. 16 illustrates an example of viewing history information collectedby the viewing history collecting unit 120 of FIG. 15. In FIG. 16,“service_id”, “event_id”, “start_time” and “event_name” are informationfor identifying a broadcasting station, starting time of a program, anda program name as described above with reference to FIG. 3. Further,“genre” is genre information as described above with reference to FIG.13.

“Series_id” represents a series descriptor of the EIT which is seriesinformation for determining whether a target program is a series program(serial program broadcast plural times). Further, “viewing component”and “viewing time” represent “component type” (or “tag value”) and aviewing time of a video component/audio component that a user viewed. Inthe present exemplary embodiment, content information of each contentdata includes the series information (“series_id”).

As illustrated in FIG. 16, genre information (“genre”) of a program “F1”is set to “0*0100”, and its series information (“series_id”) is set to“0*0106”, for example. As apparent from FIG. 16, a user viewedcomponents of “main video data” and “main audio data” of the program“F1” for 56 minutes, 15 seconds, viewed components of “sub video data 1”and “sub audio data 1” for 2 hours, 30 minutes, 48 seconds, and viewedcomponents of “sub video data 2” and “sub audio data 2” for 16 minutes,21 seconds.

FIG. 17 is a flowchart illustrating a processing procedure in the fourthexemplary embodiment. Here, steps illustrated in the flowchart of FIG.17 are executed between steps S503 and S505 of FIG. 5 described in thefirst exemplary embodiment.

Instep S1901, the upconversion control unit 112 reads contentinformation of upconversion-target content data (program) from thememory 117 to obtain series information (“series_id”). Then, in stepS1902, the upconversion control unit 112 refers to the viewing historyinformation stored (memorized) in the memory 117 to determine whether auser has viewed content in the same series as the obtained seriesinformation in the past.

If the user has viewed content in the same series in the past (YES instep S1902), the processing advances to step S1903. In step S1903, theupconversion control unit 112 calculates the total viewing time ofcomponents of the content in the same series that the user viewed in thepast.

On the other hand, if the user has not viewed content in the same seriesin the past (NO in step S1902), the processing advances to step S1904.Instep S1904, the upconversion control unit 112 obtains genreinformation (“genre”) of upconversion-target content data (program).

In step S1905, the upconversion control unit 112 refers to the viewinghistory information again to determine whether the user viewed a contentin the same genre as the obtained genre information, in the past. If theuser has viewed a content in the same genre in the past (YES in stepS1905), the processing advances to step S1906. In step S1906, theupconversion control unit 112 calculates the total viewing time ofcomponents of the content in the same genre that the user viewed in thepast.

On the other hand, if the user has not viewed a content in the samegenre in the past (NO in step S1905), the processing advances to stepS1909. In step S1909, the upconversion control unit 112 determines anupconversion target component and the upconversion processing order ofcomponents based on component attribute information (“component type” or“tag value”). Then, the processing advances to step S505.

After the upconversion control unit 112 calculates the total viewingtime of components of the content in the same series or genre that theuser viewed in the past in step S1903 or step S1906, the processingadvances to step S1907. In step S1907, the upconversion control unit 112selects a given number of components in the descending order of thetotal viewing time and determines the selected components asupconversion target components.

Next, in step S1908, the upconversion control unit 112 sets theupconversion processing order in descending order of the total viewingtime. Then, the processing advances to step S505.

For example, if content data having series information (“series_id”) of“0*0106” is selected as an upconversion target, the upconversion controlunit 112 refers to the viewing history information of FIG. 16 to set theprocessing order of upconversion-target components in descending orderof the total viewing time, that is, the order of “sub video data 1”,“sub audio data 1”, “main video data”, “main audio data”, “sub videodata 2”, and “sub audio data 2”.

In this example, the video component precedes the audio component, butthe audio component can precede the video component. Further, asdescribed in the third exemplary embodiment, the processing order of thevideo component and audio component can be controlled according togenre, for example. As for components with no viewing history, thecomponents can be upconverted in the order of increasing “tag value”, orthe processing order may be controlled according to genre.

As described above, according to the fourth exemplary embodiment, theupconversion-target component and the upconversion processing order aredetermined based on a viewing history of a user, so that theupconversion processing can be controlled according to user'spreferences. Hence, components in the same series or genre, which arebeing viewed, are preferentially upconverted, so that a user can viewhigh-quality contents earlier.

In the fourth exemplary embodiment, components are sequentiallyupconverted one by one, but the video component and the audio componentcan be processed in parallel as described in the modified example of thefirst exemplary embodiment.

Other Exemplary Embodiments

The processing described in the above exemplary embodiments can berealized by providing a storage medium, storing program code of softwarerealizing the above-described functions, to a computer system orapparatus. By reading the program code stored in the storage medium witha computer (or a CPU or a microprocessor unit (MPU)) of the system orapparatus and executing them, the functions of the above-describedexemplary embodiments can be realized.

In this case, the program code read from the storage medium realizes thefunctions according to the exemplary embodiments, and the storage mediumstoring the program code constitutes the present invention. The storagemedium, such as a floppy disk, a hard disk, an optical disk, amagneto-optical disk and the like can be used for providing the programcode. Also, compact disc read only memory (CD-ROM), compact discreadable (CD-R), a magnetic tape, a non-volatile memory card, ROM, andthe like can be used.

Furthermore, the functions according to the above exemplary embodimentsare realized not only by executing the program code read by thecomputer. The present invention also includes a case where an operatingsystem (OS) or the like working on the computer performs part or theentire processes according to designations of the program code andrealizes the functions according to the above exemplary embodiments.

Furthermore, the program code read from the storage medium can bewritten in a function expansion card which is inserted into the computeror in a memory provided in a function expansion unit which is connectedto the computer. Thereafter, a CPU or the like contained in the functionexpansion card or unit can perform part or the entire processesaccording to designations of the program code and can realize thefunctions of the above exemplary embodiments. A program embodying thepresent invention may be provided by itself or may be carried on or by acarrier medium. The carrier medium may be a recording medium or atransmission medium. The transmission medium may be a signal, in whichcase a program embodying the invention may be supplied via a networksuch as the Internet.

As described above, one embodiment of the present invention can providea signal processing apparatus (100) that upconverts content dataincluding a plurality of components including at least one of a videocomponent and an audio component, comprising: a determination unit (112)configured to perform at least one of determination processing to selectan upconversion-target component from among the plurality of componentsbased on attribute information of the content data and determinationprocessing to determine processing order in which the components areupconverted; and a processing unit (113) configured to upconvert thecomponents constituting the content data based on a result of thedetermination processed by the determination unit.

Another embodiment of the present invention can provide a signalprocessing apparatus (100) that upconverts content data including aplurality of components including at least one of a video component andan audio component, comprising: a setting unit (114) configured toperform at least one of setting processing to receive a user'sinstruction and select an upconversion-target component from among theplurality of components in response to the instruction and settingprocessing to set processing order in which the components areupconverted; and a processing unit (113) configured to upconvert thecomponents constituting the content data based on the setting processedby the setting unit.

Another embodiment of the present invention can provide a signalprocessing apparatus (100) that upconverts content data including aplurality of components including at least one of a video component andan audio component, comprising: a storage unit (117) configured to storeviewing history information regarding viewed content data; adetermination unit (112) configured to perform at least one ofdetermination processing to select an upconversion-target component fromamong the plurality of components based on the viewing historyinformation stored in the storage unit and determination processing todetermine processing order in which the components are upconverted; anda processing unit (113) configured to upconvert the componentsconstituting the content data based on a result of the determinationprocessing with the determination unit.

Another embodiment of the present invention can provide a method forupconverting content data including a plurality of components includingat least one of a video component and an audio component, comprising:performing at least one of determination processing to select anupconversion-target component from among the plurality of componentsbased on attribute information of the content data and determinationprocessing to determine processing order in which the components areupconverted; and upconverting the components constituting the contentdata based on a result of the determination processing.

In one embodiment the attribute information of the content data includesinformation about a type of each component constituting the contentdata, and the method further comprises performing the determinationprocessing based on the type of each component.

In one embodiment the attribute information about a type of eachcomponent indicates any one of a main component and a sub component, andthe method further comprises performing the determination processing soas to select the main component as an upconversion-target component.

In one embodiment the attribute information about a type of eachcomponent indicates any one of a main component and a sub component, andthe method further comprises performing the determination processing soas to determine the processing order to upconvert the main componentprior to the sub component.

In one embodiment the attribute information of the content data includesinformation about a genre of the content data, and the method furthercomprises performing the determination processing based on the genre ofthe content data.

Another embodiment of the present invention can provide a method forupconverting content data including a plurality of components includingat least one of a video component and an audio component, comprising:performing at least one of setting processing to receive a user'sinstruction and select an upconversion-target component from among theplurality of components in response to the instruction and settingprocessing to set processing order in which the components areupconverted; and upconverting the components constituting the contentdata based on the setting processing.

Another embodiment of the present invention can provide a method forupconverting content data including a plurality of components includingat least one of a video component and an audio component, comprising:storing viewing history information regarding viewed content data;performing at least one of determination processing to select anupconversion-target component from among the plurality of componentsbased on the stored viewing history information and determinationprocessing to determine processing order in which the components areupconverted; and upconverting the components constituting the contentdata based on a result of the determination processing.

In one embodiment the viewing history information includes informationabout a component selected from the plurality of components in viewing,and the method further comprises determining the component selected fromthe plurality of components in viewing, as the upconversion-targetcomponent based on the viewing history information or determining theprocessing order so as to upconvert the component selected from theplurality of components in viewing.

In the preceding embodiments the determination/setting processing is oneof (a) selecting a target component; and (b) determining the order ofprocessing. However, any kind of planning or control of the upconversionprocessing could be carried out advantageously in accordance with thepresent invention, for example how much processing resource to allocateto each different component or how much upscaling to apply to eachcomponent.

While the present invention has been described with reference to theexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all modifications, equivalent structures, and functions.

This application claims priority from Japanese Patent Application No.2007-108340 filed Apr. 17, 2007, which is hereby incorporated byreference herein in its entirety.

1. Upconversion apparatus, for upconverting content data including twoor more components capable of reproduction by reproduction apparatus,the upconversion apparatus comprising: upconversion control meansoperable to generate upconversion control information for said contentdata; and processing means operable to upconvert one or more componentsof the content data in accordance with the generated upconversioncontrol information.
 2. Upconversion apparatus according to claim 1,wherein the upconversion control means are operable to inhibit theprocessing means from upconverting one or more components included inthe content data.
 3. Upconversion apparatus according to claim 1,wherein the upconversion control means are operable to determine anorder in which components of the content data are to be upconverted bythe processing means.
 4. Upconversion apparatus according to claim 3,wherein the order is such that a component of the content data that hasa higher probability of being used for such reproduction is upconvertedbefore another component that has a lower probability of being used forsuch reproduction.
 5. Upconversion apparatus according to claim 4,wherein the upconversion control means are operable to receiveinformation relating to the content data and to generate theupconversion control information automatically in dependence upon thereceived content-data information.
 6. Upconversion apparatus accordingto claim 5, wherein the received content-data information is or includescomponent attribute information representing one or more predeterminedattributes of the components of the content data concerned. 7.Upconversion apparatus according to claim 5, wherein the receivedcontent-data information is or includes program information relating toa program.
 8. Upconversion apparatus according to claim 7, wherein theprogram information is or includes genre information representing agenre of a program, and the upconversion control means are operable togenerate different such upconversion control information for differentsuch genres.
 9. Upconversion apparatus according to claim 1, wherein theupconversion control means are operable to receive an input from a-userand to generate the upconversion control information in dependence uponthe received user input.
 10. Upconversion apparatus according to claim1, wherein the upconversion control means are operable to generate theupconversion control information in dependence upon past reproductionbehaviour of a user.
 11. Upconversion apparatus according to claim 1,wherein at least one said component is a video component and at leastone other said component is an audio component.
 12. Upconversionapparatus according to claim 1, wherein at least two said components arevideo components with different respective viewing angles and/or fromdifferent respective cameras.
 13. Upconversion apparatus according toclaim 1, wherein at least two said components are audio components indifferent respective languages.
 14. Upconversion apparatus according toclaim 1, wherein the content data includes a plurality of componentsincluding at least one of a video component and an audio component, theupconversion apparatus comprising: determination means configured toperform at least one of determination processing to select anupconversion-target component from among the plurality of componentsbased on attribute information of the content data and determinationprocessing to determine processing order in which the components areupconverted; and wherein the processing means are configured toupconvert the components constituting the content data based on a resultof the determination processed by the determination means. 15.Upconversion apparatus according to claim 14, wherein the attributeinformation of the content data includes information about a type ofeach component constituting the content data, and the determinationmeans are operable to perform the determination processing based on thetype of each component.
 16. Upconversion apparatus according to claim15, wherein the attribute information about a type of each componentindicates any one of a main component and a sub component, and thedetermination means are operable to select the main component as anupconversion-target component.
 17. Upconversion apparatus according toclaim 15, wherein the attribute information about a type of eachcomponent indicates any one of a main component and a sub component, andthe determination means are operable to determine the processing orderso as to upconvert the main component prior to the sub component. 18.Upconversion apparatus according to claim 14, wherein the attributeinformation of the content data includes information about a genre ofthe content data, and the determination means are operable to performthe determination processing based on the genre of the content data. 19.Upconversion apparatus according to claim 1, wherein the content dataincludes a plurality of components including at least one of a videocomponent and an audio component, the upconversion apparatus comprising:setting means configured to perform at least one of setting processingto receive a user's instruction and select an upconversion-targetcomponent from among the plurality of components in response to theinstruction and setting processing to set processing order in which thecomponents are upconverted; said processing means being configured toupconvert the components constituting the content data based on thesetting processed by the setting means.
 20. Upconversion apparatusaccording to claim 1, wherein the content data includes a plurality ofcomponents including at least one of a video component and an audiocomponent, the upconversion apparatus comprising: storage meansconfigured to store viewing history information regarding viewed contentdata; and determination means configured to perform at least one ofdetermination processing to select an upconversion-target component fromamong the plurality of components based on the viewing historyinformation stored in the storage means and determination processing todetermine processing order in which the components are upconverted; saidprocessing means being configured to upconvert the componentsconstituting the content data based on a result of the determinationprocessing by the determination means.
 21. Upconversion apparatusaccording to claim 20, wherein the viewing history information includesinformation about a component selected from the plurality of componentsin viewing, and the determination means are operable to determine thecomponent selected from the plurality of components in viewing, as theupconversion-target component based on the viewing history informationor to determine the processing order so as to upconvert the componentselected from the plurality of components in viewing.
 22. A method ofupconverting content data including two or more components capable ofreproduction by reproduction apparatus, the method comprising:generating upconversion control information for said content data; andupconverting one or more components of the content data in accordancewith the generated upconversion control information.
 23. Acomputer-readable medium storing a computer-executable program forimplementing a method according to claim 22.